Complexity measures, emergence, and multiparticle correlations 



o 

(N 

< 



Tobias GallcQ 

Theoretical Physics, School of Physics and Astronomy, 

The University of Manchester, Manchester M13 9PL, United Kingdom 

Otfried GuhneQ 

Naturwissenschaftlich-Technische Fakultdt, Universitdt Siegen, 

Walter-Flex-Strafle 3, 57068 Siegen, Germany and 

Institut fur Quantenoptik und Quanteninformation, 

Osterreichische Akademte der Wissenschaften, Techntkerstrafie 21, 6020 Innsbruck, Austria 

(Dated: April 3, 2012) 

We study correlation measures for complex systems. First, we investigate some recently proposed 
measures based on information geometry. We show that these measures can increase under local 
transformations as well as under discarding particles, thereby questioning their interpretation as a 
quantifier for complexity or correlations. We then propose a refined definition of these measures, 
investigate its properties and discuss its numerical evaluation. As an example, we study coupled 
logistic maps and study the behavior of the different measures for that case. Finally, we investigate 
other local effects during the coarse graining of the complex system. 
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I. INTRODUCTION 

The science of complex systems is an active area of 
research in physics and in adjacent disciplines. Many 
systems in physics, biology, the social sciences and 
in economics fall under its umbrella, and research 
into these topics is becoming increasingly interdisci- 
plinary. The quantification of complexity, however, is 
not striaghtforward and there are many approaches 
to quantify complexity in more mathematical terms, 
see for example Refs. [ll-Q or Ref. 0, where a number 
of complexity measures are listed and discussed. 

A possible way to quantify the complexity of a 
correlated multiparticle system is to take its distri- 
bution in state space and consider its distance to 
the distribution of an uncorrelated system, measured 
for example by the multi-information or excess en- 
tropy [ifl, Ull- As recently demonstrated, this con- 
cept can be generalised to higher-order multi-particle 
correlations using methods of information geometry 
[lOl \\% Il3j . For a given A^-particle system one can 
consider the space 8k of A^-partite distributions which 
are thermal states of fc-particle Hamiltonians, and ask 
what the distance of a given A^-particle distribution is 
from thi s sp ace. These ideas were discussed in detail 
in Ref. [lO|, and an appropriate mathematical defi- 
nition in terms of exponential families was devised. 
The authors of Ref. [lOl have also made available an 
numerical algorithm with which to compute the com- 
plexity measures they introduce |14l |. 

While the above mentioned approaches deal with 
classical complex systems, correlations play also a 



vital role in quantum multi-particle sysems. Conse- 
quently, many studies have been devoted to the char- 
acterization of quantum correlations |15l - [l9| . Con- 
cepts from (classical) information geometry were ex- 
tended to the quantum realm by Zhou [16|, ll8|, in 
particular a quantum analog of the above exponential 
families E^ was introduced. Based on these concepts 
the author of Ref. [16] arrives at the somewhat coun- 
terintuitive observation that local operations on a sin- 
gle particle can increase the overall correlation in a 
quantum state. More precisely, Zhou gives an explicit 
example of a thermal quantum state of a two-particle 
Hamiltonian on three particles, where a local opera- 
tion on one of the three qubits converts the state into 
one which can no longer be written as a thermal state 
of a two-particle Hamiltonian, but which instead has 
three-particle correlations^. 

One of the main motivations for the present work 
is the question whether or not similar effects can be 
seen in classical complex systems. To this end we 
characterize the class of local transformations, acting 
on single particles, and investigate how multi-particle 
correlation, as defined in Ref. [10] is affected by those 
transformations. We show that (i) local transfor- 
mations can turn a distribution generated by a two- 
particle Hamiltonian into one with genuine three par- 
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^ In the following, statements such as 'a state has fc-particle 
interactions' will always be taken to mean that the state 
cannot be written as the thermal (Gibbs) state of a Hamil- 
tonian with k — 1-particle interaction alone, but that instead 
any Hamiltonian generating the state necessarily has at least 
one fc-particle term. The absence of fc-particle correlations 
in this sense is not equivalent to a factorization requirement 
on fc-particle correlation functions often considered in sta- 
tistical physics. 



tide correlation, and as a corollary (ii) that inte- 
grating out individual particles can increase multi- 
particle correlation as well. 

In order to provide a remedy for these undesired 
properties we propose the concept of a 'local orbit' of 
an exponential family. The local orbit of a set S of 
distributions is here the set of all distributions that 
can be generated from elements of S by applying lo- 
cal transformations. We investigate the possibility to 
define multi-particle complexity not in terms of dis- 
tances from exponential families themselves, but from 
their local orbits, and we propose a numerical scheme 
with which these complexity measures can be approx- 
imated. Finally we discuss a second type of local in- 
tervention, and study how different coarse-graining 
procedures can affect the multi-particle correlation 
properties of a system of coupled chaotic maps. 

The remainder of this paper is structured as fol- 
lows: In Sec. In] we provide the basic concepts of infor- 
mation geometry and briefly describe the complexity 
measures introduced in Ref. [10]. The focus of Scc. lIIII 
is the concept of local transformations. We first in- 
troduce their mathematical definition, and then show 
how local transformations can increase the correla- 
tion content of multi-particle systems. In Sec. II VI we 
propose an alternative set of complexity measures, in- 
variant under local transformations. We also describe 
a numerical method with which to approximate these 
measures. In Sec.|V]we apply these concepts to a sys- 
tem of coupled chaotic maps, before we conclude in 
Sec. Ell Details of the numerical procedures are out- 
lined in the Appendix. 



II. COMPLEXITY MEASURES FROM 
INFORMATION GEOMETRY 

In this section, we briefly review the complexity 
measures used in Ref. [iO|. The underlying idea is 
the following: Consider a dynamical system com- 
posed of N particles each of which can be observed 
to be in one of two different states, referred to as 
and 1 in the following. At any given time the state 
<r = (cTi, ..., cr^v) of the system is therefore found to 
be an element of {0, 1}^. The space of all 2^ possi- 
ble configurations cr = (cri, . . . , ctat) will be denoted 
as n := {{ai,...,(TN),ai e {0,l},i = 1, . . . ,N} = 
{0, 1}^. The stationary distribution of the system is 
then a probability distribution over 51. 

Given such a distribution, P{-), one can ask 
whether or not it is the thermal state of fc-particle 
Hamiltonian, < fc < iV, i.e., whether one can 
write P(cr) = Z-iexp[if('=)(o-)], where H<^^\eT) is 
a Hamiltonian containing only j-particle terms with 
j — 1, . . . , fc, and where Z is a constant ensuring nor- 
malisation. Trivially, any A'^-particle distribution can 
be written as the Gibbs state of an TV-particle Hamil- 
tonian, but for fc < iV this may not be the case. One 



can then use the distance Dk (quantified by the rel- 
ative entropy) from the set of all distributions gener- 
ated by fc-particle interactions as a measure of com- 
plexity. In order to make this idea mathematically 
precise, we first give the definition of exponential fam- 
ilies, their closure, and the relative entropy. We here 
follow the work and notation of Ref. lO]. We then 
discuss some properties of the resulting distance mea- 
sures in Sec. IIIBI 



A. Exponential families 

We will write the set of all particles as F = 
{1, . . . , A^} in the following. Given a subset A C V, 
we write Ha{-) for Hamiltonians (or, more generally, 
arbitrary functions) which only depend on the {ui} 
with i € A, i.e. Ha — HA{{<Ji}i^A)- A fc-particle 
Hamiltonian (fc < N) is then a Hamiltonian contain- 
ing only terms which each depend on (at most) fc 
spins, so it is of the form 



H{a) 



E 

AcV:\A\=k 



HA{{cr^}ieA)- 



(1) 



We note that any fc-particle Hamiltonian is also an £- 
particle Hamiltonian for all £ > fc. The vector space 
of all fc-particlc Hamiltonians will be denoted by Qk 
in the following, and accordingly we have Qk C Qe 
for all i > k. 

The set of thermal states (or Gibbs measures) gen- 
erated by fc-particle Hamiltonians then constitutes a 
so-called exponential family. We write 

(2) 
where 'P(ri) denotes the set of all probability distri- 
butions over Vt. We then have 



£1 Cfs C •■• <^£n = V{9:). 



(3) 



We will first discuss some of the properties of the 
families £k- First, it is important to note that the 
probability distribution in £1 are simply product dis- 
tributions factorizing over single particles, i.e. they 
are distributions which can be written as P{<t) = 
P(i)(cri)p(2)(cr2) • ... • P'^^Kun) with single-particle 
distributions P^'^'{-). The distributions in the more 
complex exponential family £2 on the contrary can 
be written as products of the form 

P{a)=M n ^^^''('^-'^.), (4) 

where the {p('-')} are two-particle distributions and 
where the constant Af provides appropriate normal- 
ization. Factorizations of this type can be extended 
straightforwardly to describe the elements of £k as 



products of A:-particle distributions. There are other 
equivalent characterizations of ffc, we wih discuss and 
use some of them later in Section IIIIBI 

Given that Hamiltonians assign finite energies to 
all configurations cr, i.e. H{cr) G M for all H € Qk, 
k — 0, . . . ,N and all cr e fi the probability distribu- 
tions Sk by construction carry non-vanishing proba- 
bility for all cr G il., i.e. P(cr) > for all cr and all 
P € 8k- It is therefore useful to consider the closure 
£k to incorporate probability distributions without 
full support. For example, the three-particle prob- 
ability distribution P with P(OOO) = P(lll) = 1/2 
and P(-) = elsewhere is not in £2, but in 82, since 
it can be approximated by the low-temperature limit 
of an Ising-type two-particle interaction. The gen- 
eralization of this distribution to more than three 
particles can still be generated with two-particle in- 
teractions only. On the other hand, the iV-particle 
distribution P with 

r 1/2^-1 if Y.^^^=^ mod 2, 

P{ct) = <^ (5) 

I elsewhere 

cannot be generated via N — 1-particle interactions, 
but instead requires a Hamiltonian with A'^-particle 
interaction. This distribution assigns probability 
2-(A''-i) |-Q g^jjy i^itstring <t with an even number of 
bits equal to one, and zero to all other bitstrings. If 
only A^ — 1 entries in <y are known, it is not possible 
to tell whether or not this condition is met, hence one 
cannot decide whether P(cr) = 2^^^^^^ or whether 
P(cr) = 0, this requires always knowledge of the state 
of all particles [see also point (iii) below] . 



B. Distance measures 

Given a probability distribution P(-) over il and 
an interaction order fc = 0,...,A'^ — 1, wc then ask 
how closely P(-) can be approximated by distribu- 
tions generated by fc-particle Hamiltonians. This is 
captured by the following 'distance' 



Dk(P) 



inf P'(P||Q), 
Qe£fc 



(6) 



where D{P\\Q) is the KuUback-Leibler distance [2| 

P{a) 



D{P\\Q)= > PMlog2 



cr&i 



Q{(t)' 



(7) 



This quantity has the following properties: 

(i) Since P'(P||Q) = if and only if P = Q, the 
quantity Dk{P) is non- vanishing for all distri- 
butions P ^ Ek- 

(ii) The minimizing distribution Q* 6 £k on the 
RHS of Eq. (|6]) is given by the maximum like- 
lihood approximation of P by distributions in 
£fc, see Refs. [lfl[l3 for further details. 



(iii) A second characterization of the minimizer Q* 
is the following: Q* is the distribution of max- 
imal entropy in the set of distributions with 
the same fc-particle marginals as P [12]. This 
implies that the distribution from Eq. (IS|) has 
Dk{P) = 1 for all k < N, since the flat distri- 
bution has the same k — 1-particle marginals as 
P and clearly maximizes the entropy over all 
distributions. 

(iv) For the case fc = 1 the minimizer Q* can di- 
rectly be found as 



Q*{rT) = P^''iai)P^'>{a2) ■ ... • P^^^^(ajv), (8) 

where the p(*)(-) are the single-particle 
marginals of P. The quantity Di is also known 
as multi-information 12, 21], the above quan- 
tities Dk can be considered as a generalization 
of this concept. 

(v) For fc > 2 an analytical calculation of Dk{P) 
for a given distribution P £ P(^) is usually 
not straightforward. There are, however, pow- 
erful numerical tools for its computation [lj|, 
for completeness we outline a possible algorithm 
in the Appendix. 

(vi) Following Ref. [ifl] one can also consider 

I^''\P) - Dk-i{P) - DkiP), k = l,...,N (9) 

as complexity measures. The quantity Dk{P) 
here represents the improvement in approximat- 
ing P S V{^) when fc-particle terms in the gen- 
erating Hamiltonian are allowed over the case 
in which only k — 1-particle interaction is ad- 
mitted. In this paper, however, we will mainly 
work with Dk{P)- 

(vii) Finally, possible generalizations to the quantum 
setting have been discussed in Refs. |16l-[l8l|. 

We also note that the choice of the KuUback- 
Leibler divergence [Eq. ([§])] is of course not the only 
choice of an underlying distance between probabil- 
ity distributions. Other distance measures are con- 
ceivable, see for example Refs. (a-Q- In our work 
we will however restrict the discussion to the relative 
entropy, as this provides several useful properties of 
D(-||-) which we will use later on. 



III. LOCAL TRANSFORMATIONS 

A. Definition and interpretation 

We will next discuss the behavior of the measure 
Dk under local transformations. In considering lo- 
cal transformations we have the following scenario in 
mind (see also Fig.[T]): a complex system composed of 



Alice 



Charlie 




FIG. 1: (color online). Illustration of the effect of local 
transformations: A complex interacting dynamics gener- 
ates a probability distribution over the state space of four 
binary particles. Each observer (Alice, Bob, Charlie and 
Dave) has access to the state of one particle, and reports 
these to a central authority (not shown). One type of lo- 
cal transformation arises, if one party, say Alice, changes 
the information she transmits stochastically, but with- 
out communication with the other observers (see text for 
further details) . A second possible type of local transfor- 
mation is given by disregarding one party, say Dave, and 
considering only the marginal distribution on the remain- 
ing three parties (this corresponds to a case in which Dave 
randomly reports to have seen state or 1 regardless of 
what he actually observed). It is natural to require from 
a measure of complexity that it should not increase under 
either type of local transformations. 



N binary particles generates successive states cr{t) = 
[(Ti{t),(T2{t), . . . ,aN{t)], where t — 1,2,... denotes 
time. The resulting stationary distribution^ over fl = 
{0, l}''^ is then given by the cumulative histogram of 
all observed A^-particles states in the long-time limit, 
i.e. P{(t) = limt_j.oo t~^ J2t'=i ^(TCT{t')^ virhcrc 6 is the 
standard Kronecker symbol, i.e. 5aai = 1 if cr = cr' 
and 6(j(T' — otherwise. Computing this distribu- 
tion however requires a centralized observer, who has 
access to the sequence (T{t). 

To introduce the concept of local transformations 
we now consider a different scenario and assume that 
there are N distant observers who each have access to 
sequence of states of only one of the N particles, and 
then report these to a centralized agent who aggre- 
gates the information. By local transformations we 
mean manipulations a single observer can execute on 
the state of the particle he (or she) has observed be- 
fore passing on the information to the centralized site. 
Assume for example that observer i G {!,..., TV} 
upon observation of state reports to have seen state 
with probability 1 — a^ and that he (or she) reports 



state 1 with probability ai in such circumstances. 
Similarly if presented with observation 1 he or she 
may report state 1 to the central site with probabil- 
ity 1 — &i, but pass on wrong information with rate 
hi {ai,bi S [0,1]). The central site then accumulates 
the information and obtains an A'^-particle distribu- 
tion P{-), computed from the reported data of the N 
observers. 

Unless P itself is pathological one will generally 
have P = P only if a^ — bi = for all i. Behaviour 
on the part of the observers in which this is not the 
case can be thought of as noise (i.e. inaccurate obser- 
vation) or as deliberate falsification of the observed 
input. The {ai^bi} are error rates, quantifying how 
frequently inaccurate information is transmitted to 
the central agent. The crucial point is that such ma- 
nipulations occur locally, i.e. different observes may 
not collude with each other and communicate before 
reporting to the central site. All operations therefore 
act on single particles only. It appears reasonable to 
ask that such transformations should not increase the 
complexity of P(-): if under a given measure of com- 
plexity fc-particle correlations are absent in P{-) then 
applying local transformations should not introduce 
fc-particle correlation in P (the effective distribution 
perceived by the aggregating central agent). In a sim- 
ilar vein disregarding one or several particles should 
not introduce additional correlation: if one considers 
the marginal probability distribution of the states of 
a subset of M < A^ particles, one may require that 
these M-particle marginals should be not more com- 
plex than the joint probability distribution of all A'' 
particles. Similar considerations are well known from 
quantum information theory, where a basic require- 
ment for entanglement measures is that they should 
not increase under local operations and classical com- 
munication. 

In order to investigate whether the complexity 
measures defined above do indeed respect these con- 
straints we write Aj^'^i^. for the probability with which 
observer i reports state /i^ S {0, 1} to the central site 
when he/she has in fact seen state Vi € {0, 1}. We 



We will 



always have ^ Afii.fi = 1 for all i and 

use the notation A*-^^ = (^^ "'b') in the following. A 
specific local transformation is then characterized by 
the parameters uj — (oi, 6i, . . . , cat, ^a?) € [0,1]^''^, 
we will often use Tu; as a short-hand for the transfor- 
mation defined by u:. 

For any local transformation T^""^ = Tuj the result- 
ing distribution P(-) put together by the central site 
can then be written as 



^(m) = E^/x°^^M, 



V 



where 



^ We here only consider interacting systems which do have a 
stationary distribution. 



^ ^ / 1 h 

loc _ /0\ A(i) _ /0\ ( t — fti Oi 

a,. 1 - b,. 






(10) 



(11) 



Equivalently the probability of the central site con- 
structing the A^-particle configuration fi from the re- 
ports he or she receives from the N observers when 
the true configuration of the iV-particle system was 
v is given by 



N 



rploC 



u^. 



(12) 



The superscript 'loc' here indicates that we are in- 
terested in local transformations, i.e. those for which 
the matrix T has the product structure indicated in 
Eqs. (fm[T2)) . More generally we can consider linear 
transformations Q(/i.) = J^i/^fJ'i^^i'^)- These are 
well defined so long as the entries of the matrix Tuv 
are nonnegative and the entries in the columns add 
up to one, J2ij, Tfiv = 1, i.e. so long as the matrix T 
is stochastic^. 

In order to probe the measures of correlations in- 
troduced earlier we next investigate how the distances 
Dk behave under local operations. As it turns out, 
distances from exponential families can indeed in- 
crease under local transformations. 



B. Local operations can increase Dk 

To give an example, first note that under arbitrary 
transformations (not necessarily local) the Kullback 
Leibler distance can only decrease, i.e., D{P\\Q) > 
D[T{P)\\T{Q)] '2S], where we have written T{P) for 
the image of P(-) under a transformation mediated 
by the matrix T [and similar for T(Q)]. As a conse- 
quence, the following two statements are equivalent: 

(1) Dk decreases under local transformations, i.e. 
Dfc[T'°'=(P)] < Dk{P) for aU P and all local 
transformations T'°'^, 

(2) the manifold £k is invariant under local trans- 
formations, i.e. T^°'^{Q) e £k for all Q E £k and 
all T^°'=. 

As we will now show neither of these two statements 
is true however. 

In order to construct a counterexample it is use- 
ful to detail further the characteristics of probability 
distributions in Sk- For simplicity, we focus on the 
case of three particles and consider £2, generalization 
to the other cases is straightforward. It turns out to 
be helpful to formulate the problem in the language 



^ If one requires in addition that the row sums are normahzed, 
y^,, T/xiy = 1 the matrix is called doubly stochastic and the 
entropy increases during the process. Doubly stochastic ma- 
trices naturally occur, if one requires that the maximally 
mixed distribution is invariant under the transformation. 



of quantum mechanics. In analogy to the space con- 
sisting of the 2^ configurations {cr} = {(ci, ■■■as)} = 
{(0,0, 0), (0, 0, 1), ..., (1, 1, 1)} we consider analogous 
quantum states in the orthonormal computational 
basis, {\(t)} = {|000},|001),...,|111)}. For a given 
probability distribution P{cr) we then consider the 
Hamiltonian defined by 



h=j:x. 



k)('^l, 



(13) 



where the {Xcr} are given by Xcr — InP(cr). For 
the moment we restrict the discussion to probabil- 
ity distributions P with full support, i.e. we as- 
sume P{cr) > for all cr. The operator H is then 
well defined and diagonal in the computational basis, 
H\cr) = X(j-\cr), and we have 



P{<T) = {a\cxp{H)\a). 



(14) 



Using the fact that tensor products of Pauli matri- 
ces form a basis of the space hermitian operators, 
any Hamiltonian of the type defined in Eq. (J13p can 
uniquely be written in the form 



.(3) 



0) 



-70-^^-' cr^^-' (K) <7l 



(15) 



.(») 



where al denotes the Pauli matrix az acting on 
qubit i. We will in the following generally not treat k 
in Eq. (|T5|) as free parameter, but choose it such that 
the distribution P(-) constructed from H via Eq. P^ 
is correctly normalised. 

A Hamiltonian of the type described in Eq. ((15]) 
is a two-particle Hamiltonian if and only if the 3- 
particle term is absent, i.e. if 7 = 0. This is the case 
if 



tr 



H 



r(l) 



r(2) 



.(3) 



0. 



(16) 



This in turn is equivalent to Aqoo + Aoii+Aho-I-Aioi = 
Aooi + Aoio + Aioo + Am. So any probability dis- 
tribution P € £2 fulfils ln[P(000)] + ln[P(011)] + 
ln[P(110)] + ln[P(101)] = ln[P(001)] + ln[P(010)] + 
ln[P(100)] +ln[P(lll)], sec also Rcf. [12]. Equiva- 
lently, one has 

P(OOO) • P(Oll) ■ P(llO) • P(lOl) 
= P(001)-P(010)-P(100)-P(lll). (17) 

These considerations can now be used to show that 
the above two statements (1) and (2) do not hold, 
and that instead local transformations do not leave 
£2 invariant. To this end one use Eq. P^ to gen- 
erate two-particle Hamiltonians and their associated 
probability distributions P G 1^2 at random and then 
subsequently apply a local transformation T^°'^ with 



randomly chosen matrix elements A 



(*) 



Relation 



P?)) can then be used to check whether or not the 
outcome is still in £2- It turns out that generally 
T^°'^{P) ^ £2- This proves that D2 can increase un- 
der local operations. 

Our considerations complement those of Ref. [lg| , 
where it was observed that a quantum analog of the 
quantity Dk can increase under local operations and 
classical communication. In the quantum case, how- 
ever, the possible local transformations form a much 
larger class of maps: For instance, the positive maps 
on a single qubit are parameterized by 12 parame- 
ters, while in the classical case, a transformation on 
a single particle has only two degrees of freedom. Our 
example shows that the increase of Dk occurs already 
in the classical regime and is also not due to possi- 
ble non-commuting terms in the Hamiltonian in the 
quantum setting. 

Finally, we would like to stress that the manifold 
£1 of distributions factorizing over individual parti- 
cles is clearly invariant under local transformations. 
Hence, the quantity Di (also referred to as multi- 
information 12, 2V1) does not increase under local 
transformations. 



C. Tracing out particles can increase Dk 

A second requirement one may ask of correlation 
measures is that they do not increase when individ- 
ual degrees of freedom are integrated out. Math- 
ematically formulated, the question is whether the 
M-particle marginals (M < N) of a an TV-particle 
distribution in £k are still generated by a fc-particle 
Hamiltonian. For our later discussion it is important 
to realize that tracing out individual particles can 
be understood as carrying out a local transformation 
with specific parameters. More precisely, one may 
for example choose to apply the local transformation 
defined by 



A« = 



1 — fli 



ai 
1 — fli 



(18) 



the original distribution P(-), i-e., P'{<J2, ■ ■ ■ ^ctn) = 
Eai Pi'^1,'^2, . . . , ffw). Thus T'°'{P) E £k if and only 
iiP'e£k. 

Again, one can directly find counterexamples, 
which show that Dk can increase under tracing our 
particles. For instance, if one generates a random 
four-particle distribution in £2, then, after tracing 
out one particle, the marginal distribution is gener- 
ally not found to be in £^2- 



IV. INCORPORATING LOCAL 

TRANSFORMATIONS IN THE 

CORRELATION MEASURE 

In the last section we have seen that the measures 
Dk can increase under local transformations, and that 
they are hence lacking a desirable property of corre- 
lation measures. To overcome this, we may replace 
the manifolds £k by their local orbits. 



Ck = {T{P)\TeT'°',Pe£k}- 



(21) 



The notation T'"'^ here indicates local transforma- 
tions of the type defined in Eqs. (|llll2p . The set £k 
is manifestly invariant under local transformations, 
hence the quantity 



C,(P) =inf DiPWQ) 



(22) 



and 



is a correlation measure which is nonincreasing under 
local transformations. It is also nonincreasing under 
tracing out particles, as this is a special case of a lo- 
cal transformation. Given the invariance of £1 under 
local transformations, the quantity Ci coincides with 
the multi-information. 

In the following we will first discuss some basic 
properties of this measure, before we will describe 
how to approximate Ck numerically. In the next ses- 
sion we will then apply this measure of complexity 
to the example of a set of coupled chaotic maps, we 
will also provide a comparison with existing results 
on the measure Dk [1(]| . 



Ai'') 



1 
1 



(19) 



for k = 2, ..., N. For the first particle reports state 
to the central site with probability 1 — ai, and state 1 
with probability oi irrespective of the actual state of 
the particle. The states of the remaining particles are 
reported faithfully. If applied to an A^-particle distri- 
bution P(-) the transformation T'°'= = 0,^^ A^*) will 
result in a distribution of the form 

T'°%P){a) = R{ai)P'{a2, . . . , a^) (20) 



where i?(o'i = 0) = 1 
and where P'(-) is the 



- ai and i?((Ti = 1) = ai, 
N — 1-particle marginal of 



A. Properties of the set £k and the measure Ck 

Let us first discuss some properties of Ck- First we 
observe that for A^ > 4 particles C2 is a set of mea- 
sure zero in the space of all probability distributions. 
To see this one first notes that £k is characterized by 
Si=i ( i ) fr^^ parameters, this is the number of possi- 
ble terms in a Hamiltonian up to and including order 
k (sec Section fill Bl) . Furthermore, local transforma- 
tions are parameterized by two variables per particle, 
i.e. by 2N parameters in total. Given that A^-particle 
distributions carry 2^ — 1 free parameters and that 
1 > J2L1 (T) + 2iV for A^ > 5 we conclude that 
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FIG. 2: (color online). Complexity measures 1^, fc = 1, . . . , 6 as a function of the coupling parameter e for a system 
of Ai" = 6 coupled tent maps (upper six panels), and for A'' = 6 coupled logistic maps (lower six panels). Each panel 
shows three curves corresponding to coarse-graining procedures with thresholds = 0.25, 0.5 and 0.5 respectively. All 
data is obtained from running 1000 iterations of the iterative projection algorithm. Curves represent the average of Ik 
over 20 samples with independent random initial conditions Xi{t = 0) G [0, 1]. 



the set £2 is of measure zero in the space of all proba- 
bility distributions. Similarly, one can argue that the 
Ck for small k are of measure zero for many particles. 

For lower particle numbers the set C2 does not 



cover the whole space of distributions either. To see 
this, consider three particles and the probability dis- 
tribution P(-) with P(OOO) = P(Oll) = P(lOl) = 
P(llO) = 1/4, and P{(t) = otherwise. We know al- 



ready that D2(P) = 1 is relatively large, hence it 
is a natural candidate to be outside of £2- In or- 
der to show that this is indeed the case, assume 
there exists a local transformation T e 7'(i°'=) and 
a. Q & £2 such that P = T{Q). We parameterize 
T = A(i) (g) A(2) (g, A(3) via AC^) = (1^'^,'= ^) and we 
have 

i'(OOl) > Q(000)(l-ai)(l- 02)03, 

P(OIO) > Q(000)(1- 01)02(1 -as), 

i'(lOO) > Q(000)oi(l-o2)(l-O3), 

P(lll) > Q(000)oiO2O3. (23) 

The distribution P{-) is constructed such that the 
left-hand-sides of these equations vanish. Assume 
now that Q(OOO) > 0. In order to satisfy the above 
inequalities we must have o^ e {0, 1} for all k. In a 
similar manner, making the assumption (5(111) > 
leads to the requirement hk & {0, 1} for all k. 

For cases in which (5(000) and (5(111) are both 
non-zero this solves the problem, this includes in par- 
ticular all distributions Q{-) with full support. The 
local transformations are then such that any party 
applies one of the following transformations: (i) they 
do not make any modification to the state they ob- 
serve (ofe — bk — 0), (ii) they always flip the binary 
symbol they receive (ofe = 6^ = 1) or (iii) they always 
report the same result to the central site no matter 
what input they receive (e.g., if Ofe = 0, &fc = 1 they 
always report and if a^ = 1,6^ = they always 
report 1). With these transformations, however, one 
cannot obtain the probability distribution P(-) from 
a distribution in £2 [in the case (iii) this is impossi- 
ble as P assigns positive probability to bit strings in 
which the variable ak takes either of the two values 
and 1]. 

It remains to address cases for which there are <t G 
Q such that Q{cr) = (i.e. Q is in the closure £2, 
but not in £2 itself). First, using the reasoning from 
above, one can directly see that if (5(0, cr2, 173) > 
for some 0-2, 173, then oi e {0, 1} and if (5(1, <^2, o'a) > 
for some 0-2, 173, then 61 e {0,1}. This solves the 
problem for many further cases, for instance if there 
are two events <Ji,a2, u^ and a^, (T2, a'^ with ak 7^ cr^ 
and Q{Ti,a2,<J3) > as well as Q{a[,a'2,(j'^) > 0. 

The only remaining cases, where we cannot set 
any constraints on oi (or 61) are of the type where 
(5(0, 0-2, 0-3) = for all 0-2,0-3 [or (5(1,0-2,0-3) = for 
all 0-2,0-3]. But for such cases the value of oi (or 61) 
has no effect on T{Q), so, without changing the out- 
come of the transformation, such a parameter can be 
set to either or 1, again leading us to the desired re- 
sult. Finally, note that due to continuity reasons and 
the the fact that £2 is compact, P has a finite distance 
to £2- Using the algorithm outlined in the next sec- 
tion we find a numerical distance of C2{P) ~ 0.689. 



B. Numerical calculation of Ck 

It is not a-priori not straightforward to compute 
Ck{P) for a given iV-particle distribution P(-). We 
have however made some progress in devising an it- 
erative numerical scheme. We cannot prove at the 
moment that it always converges, so strictly speaking 
it provides an upper bound on Ck{P)- The quantity 
Ck (-P) as defined in Eq. ([^ is given by 



Ck{P) = inf D[P\\T{Q)l 
Q&£k 



(24) 



so we need to optimize the choice oi Q e £k and that 
of T S 'J'C'"'^) simultaneously. We here proceed iter- 
atively. Given a test distribution P(-) we first find a 
Hamiltonian H £ Qk, parameterized by a set X^^', so 
that (5\(i) is the best approximation of P in £k- As 

a next step we then find parameters uj^^> which mini- 
mize the distance D[P\\T(jj(i) ((5\(i))], i-e. we find the 
point in the local orbit of (5 \ (i) closest to P. We then 
turn to an optimization of the fc-particle Hamiltonian 
again, keeping Tj^ci) fixed while optimizing Q £ £k. 
This is to say, find a set of parameters A' •* such that 
-'^[-P||^a;(i)(Q_\(2))] is minimized (subject to the con- 
straint (5\(2) G £k)- Subsequently, we optimize the 
local transformation again, and find a new set of pa- 
rameters a;^^) minimizing -C)[-P||7a;(2) (Q_\(2))]- This 
procedure is then iterated. Further details on the 
exact implementation can be found in the Appendix. 



V. APPLICATION TO COUPLED CHAOTIC 
MAPS 

In this section we apply the proposed measures of 
complexity to a specific case of an interacting multi- 
particle system. In particular we consider a coupled 
set of TV discrete-time maps of the form [22] 



x,{t + l) = {l-e)f[xdt)] 



(25) 



where i,j = 1, . . . , N. Similar arrangements were 
studied in Ref . [101 , '^^ particular different types of un- 
derlying adjacency networks were addressed. We here 
focus on the all-to-all coupling described in Eq. (P5|) . 
The parameter e e [0, 1] describes the strength of the 
interaction between the maps. For e = the N maps 
are fully uncoupled, for e > they become increas- 
ingly more coupled. The chaotic maps we have used 
are the tent map, given by 



/(^) = 



2x 
2-2x 



< X < 1/2 

1/2 < a; < 1 ' ^^' 



logistic map 



tent map 





FIG. 3: (color online). Complexity measures Ck and Dk (k — 2,3) for sets of A'^ = 4 coupled logistic maps (left- 
hand panel), and A'^ = 4 coupled tent maps (right-hand panel). Coarse-graining is performed at threshold Q — 1/2. 
Open symbols show the distances Dk from exponential families £k of probability measures generated by fc-particle 
Hamiltonians, filled symbols are upper bounds on the distance from the local orbits of £k (see text for details). 



and the logistic map defined by 



(27) 



For both maps a symbolic dynamics of binary sym- 
bols is derived from the continuous variables Xi{t) e 
[0, 1] via a simple coarse graining procedure of the 
type 1231 



fT.(t) = 



x,{t) < e 
x^it) > e 



(28) 



The parameter Q e i^A) is here a coarse- graining 
threshold. 

To obtain the probability distribution, we start 
with a random configuration of the Xi and interate 
the coupled map 10^ times. We disregard the first 
10^ iterations and use the remaining iterations to de- 
termine our probability distribution. 



in Fig. [21 (red curves). As reported in Ref. JJiJ the 
system of coupled maps is driven toward a state of 
synchronized chaos if the coupling strength is suffi- 
ciently large {e > 0.45 for the system of tent maps, 
e > 0.2 for the logistic maps). In this regime the 
resulting distribution P(cr) is found to be in £2, and 
accordingly we have I^''> = for fc > 3. With the 
exception of 7*^^^ which is seen to be monotonically 
increasing in e all other measures I*^*^^ attain max- 
ima at e w 0.35 for the system of coupled tent maps, 
indicating that the most complex regime occurs just 
below the synchronization threshold [lO|. A similar 
observation is made for the system of logistic maps, 
see the red curves in the lower six panels of Fig. [2l 



Application of the modified complexity 
measure to coupled maps 



A. Previous results 

For completeness we will first re-iterate some of 
the results of Ref. [10] ■ Varying the value of the 
coupling strength e we run simulations of a set of 
N = 6 coupled tent and logistic maps. For the 
choice Q — 1/2 we obtain the corresponding sym- 
bolic dynamics, and measure the resulting station- 
ary distribution P{(t), where cr = (ai, . . . , erg)- We 
then use the iterative projection algorithm proposed 
in Ref. [10| (and discussed in more detail in the Ap- 
pendix), to obtain Dk{P) for A: = 0, . . . , 6. We then 
construct Ik = Dk-i—Dk (fc = 1, . . . , 6) from the out- 
come of the iterative projection. Results are shown 



Results for the modified complexity measures Ck 
are shown in Fig. [31 Realistic computing resources at 
present only allow us to study relatively small systems 
with A'' = 4 particles, as our implementation of the 
numerical computation of the improved measure Ck 
is significantly more demanding in computing time as 
that of the measures Dk ■ As the original complexity 
measure the modified measures are well able to detect 
the onset of synchronization. This is unsurprising as 
Dk =0 implies Cfc = 0. 

For the cases we have studied here the functional 
dependence of Ck broadly follows that of Dk- For 
fc = 3 in particular our numerical results for the two 
measures are essentially indistinguishable up to minor 
deviations. As seen in Fig. [3] the modified measure 
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FIG. 4: (color online). Complexity measures /^ and distances Dk (fc = 1, . . . , 6) in a system of A'^ = 6 coupled tent 
maps (upper two panels) and for TV = 6 logistic maps (lower two panels) as functions of the coarse-graining thresholds 
O. The coupling strength is fixed at e = 0.34 for the tent map and at e = 0.17 for the logistic map. All data is obtained 
single runs of the coupled maps, started at a random initial condition. The iterative projection algorithm is run for 
1000 iterations. 



of complexity can on occasion deviate substantially 
from the original one, see for example the data points 
near e = 0.1 in the case of coupled logistic maps (left- 
hand panel of the figure). It is here important to keep 
in mind that the algorithm for the calculation of Ck is 
not guaranteed to converge. This, along with the rel- 
atively high costs in computing time, are clear draw- 
backs of the complexity measure we propose here. Fu- 
ture work may therefore address improved algorithms 
for computing these measures, and/or more detailed 
comparison with the complexity measures proposed 
by [10] . Should it turn out that both sets of mea- 
sures give broadly the same results, then it may well 
be appropriate to continue to work with the Dk (or 
Ik), despite the lack of a non-increasing nature under 
local transformations. 



C. Dependence on single threshold 



As final point of our study we have investigated the 
influence of the quantitative value of the coarse grain- 
ing threshold Q on the complexity measures Dk and 
Ik- It is here important to be aware that he coarse 
graining procedure, mapping a real-valued variable 
Xi(t) in the unit interval onto a discrete degree of 
freedom ai{t) G {0, 1}, can be considered as a local 
operation as well, but it is a local procedure in the 
construction of the probability distribution, and not 
a local transformation applied after constructing the 
probability distribution."* In performing the coarse 
graining each particle is treated independently from 
the rest of the system, and hence the applied opera- 



^ Therefore, one cannot set the strict requirement for any com- 
plexity measure to be invariant under this transformation. 
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FIG. 5: (color online). Complexity measures Ck and 
Dk {k = 2, 3) for sets of A*' = 4 coupled tent maps at 
fixed coupling parameter e = 0.34 and at varying coarse- 
graining threshold Q. Open symbols show the distances 
Dk from exponential families £k of probability measures 
generated by fc-particle Hamiltonians, filled symbols are 
upper bounds on the distance from the local orbits of £k ■ 



tion is local.^ 

As seen in Fig.[5]the behaviour of the {Ik] as func- 
tions of the coupling strength can depend consider- 
ably on the choice of the local threshold. All curves, 
except for those of I2 for the case of coupled tent 
maps, show strong qualitative changes as 6 is varied. 
This is also confirmed in Fig. 2] where we show the 
0-dependence of I^ and Dk explicitly at a fixed cou- 
pling strength £, chosen below, but close to the onset 
of synchronisation. Changing the numerical value of 
the threshold can result in changes of the Ik and Dk 
by orders of magnitude, and as seen in the left-hand 
panels of Fig. H] it can also change the relative order- 
ing of the Ik , therefore affecting their interpretation 
as measures of complexity. 

In Fig. [5] we have tested the extent to which the use 
of the modified complexity measures Ck can remedy 
these effects. Again, for the three-particle correlation, 
we find that using local orbits instead of exponential 
families alone does not have any significant effect (we 
find that C3 « D3 for a broad range of values of the 
coarse- graining threshold 0). The functional depen- 
dence of C2 on the threshold parameter Q does how- 
ever appear to be weaker than that of the measure 
D2, we detect more variation of D2 for intermediate 
values of 8, where as the numerical estimate of C2 
varies only by comparably small amount. While this 
may provide some indication that the modified mea- 
sures exhibit a lesser dependence on the details of the 



^ The following procedure would constitute an example of a 
nonlocal coarse-graining: set ai = ii Xi{t)xi^i{t) < 1/2 
and ai = 1 otherwise. The expression i -|- 1 is to be taken as 
'mod n\ 



coarse-graining procedure that the original measure 
of complexity, more work is required to test whether 
this continues to hold for other systems and/or sys- 
tems with a larger number of particles 



VI. CONCLUSION 

In summary, we have investigated measures of com- 
plexity and emergence for classical interacting parti- 
cle systems. Based on concepts from information ge- 
ometry such measures have recently been proposed 
[lO| . The main idea behind these measures is to con- 
sider the distance of a given probability distribution 
from the set of distributions generated by fc-particle 
Hamiltonians, that is to say the set of distributions 
which can be factorized into a product of fc-partite 
distributions. We have shown that such measures 
are generally not invariant under local transforma- 
tions, acting on individual particles. In particular 
we have demonstrated that, somewhat counterintu- 
itively, local transformations can increase the degree 
of multi-particle complexity in this measure, generat- 
ing for example 3-particle correlations from 2-particle 
correlations. Similarly integrating out an individual 
degree of freedom can increase the complexity of the 
remaining marginal. Recent work by Zhou [l6| has 
revealed analogous findings in the quantum realm, 
our work here demonstrates that these effects are not 
intrinsically quantum, for example related to the non- 
commutative nature of quantum mechanics. Instead 
they are also seen in classical systems. 

In order to remedy these undesired properties of ex- 
isting complexity measures, we have devised a mod- 
ification, and propose to consider the distance from 
local orbits of exponential families instead of the dis- 
tance from the families themselves. These orbits are 
manifestly invariant under local transformations of 
the type we have defined, and as a consequence the 
resulting complexity measure can only be reduced by 
applying a local transformation, but not increased. 
We have devised a numerical scheme with which to 
calculate upper bounds for this complexity measure, 
results are presented for dynamical systems composed 
of multiple interacting chaotic maps. 

Finally we have investigated how the choice of lo- 
cal coarse-graining thresholds affects the complexity 
measures proposed by Ref. [10]. The coarse grain- 
ing procedure here constitutes a local manipulation 
as well, albeit different from the type we consider 
above. Numerical results indicate that the choice of 
coarse graining threshold can have significant effects 
on the resulting estimates of complexity, hence care 
needs to be taken in turning the underlying continu- 
ous degrees of freedom into symbolic language on the 
coarse-grained level. 

While the modified complexity measure we put for- 
ward here appears to exhibit more convenient be- 
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haviour under local transformation that the ones pro- 
posed in Ref. ^^ it is important for keep in mind 
that the algorithm we are able to propose produces 
upper bounds for the required complexity measures, 
but at present we are unable to decide whether or not 
it converges to the desired quantities asymptotically. 
More work along these lines is required. Additionally 
the numerical scheme we use here is limited to small 
systems, hence there is significant room for improve- 
ment. This is however beyond the scope of the cur- 
rent work which aims mainly to identify the key prop- 
erties of the complexity measures we have discussed, 
to relate them to measures based on exponential fam- 
ilies and to discuss their behaviour under local trans- 
formations. Our work is therefore mostly of a con- 
ceptual nature, and we hope our results will stimulate 
further work towards efficient algorithms for complex- 
ity measures with the local invariance propery. Ad- 
ditional future lines of research may also address ge- 
ometric concepts of distances other than those based 
on the KuUback-Leibler divergence. Finally, it would 
also be interesting to study the quantum analog of 
exponential families and their local orbits in more 
detail. Work along these lines is in progress. 
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VII. APPENDIX: ALGORITHMS 

This Appendix contains details on the two main 
algorithms used in this work. In Sec. IVII Al we pro- 
vide the precise steps with which to compute Dk{P) 
for a given distribution P, i.e. the distance from the 
set of distributions generated by fc-particle Hamilto- 
nians. In Sec. IVIIBI we explain in more detail how 
to compute an upper bound for Ck{P), the distance 
of P from the local orbit of £k ■ 



A. Iterative projection algorithm 

Ou r p resentation here closely follows that of 
Ref. [Tj], see also Ref. [10] ■ Given an (empirical) 
probability distribution P(-) on fi = {0, l}-*^, and an 
integer k < N (the order of interaction at which we 



are approximating), the algorithm to find Dk{P) is 
as follows: 

1. Computation of marginals: For each subset 
A C V with k elements {\A\ — k), compute 
the following 

uaH^ Y, P{ct'). (29) 

(J':i:A(<y')=iTA(<y) 

Here 7r^(cr) denotes the projection of cr onto 
A, i.e. TTAio-) = (cri){ieA}- The quantity aA{cr) 
only depends on the components Ui of cr with 
index i £ A^ i.e. on the fc-variables {ai)i^A- 

2. Initialisation: Initialise Q as the fiat distribu- 
tion over n-. Q{(t) = 1/(2^) for all cr. 

3. Improve current approximation: Run through 
all sets A C V with |yl| = fc and update Q(-) as 
follows: 



gnew^^^ ^ c^^^)Q^^)^ 



where 

ca(<t) 



aA{cr) 



Ea, 



TTA{(T')=TrA{Cr[ 



O(t') 



(30) 



(31) 



4. Update and iterate: Replace Q(-) -^ Q^'^^{-) 
and goto 3. 



B. Approximate scheme to calculate distance 
from a local orbit 



Given a test distribution P(-) the algorithm pro- 
ceeds as follows: 

1. Find a Hamiltonian H lE Qk, parameterized by 
a set A'^', so that Q^(i} = Z~^exp(iJ) is the 
best approximation of P in Sk- This can for 
example be done using the iterative projection 
algorithm described above. 

2. Optimization of the local transformation for 
a given Hamiltonian: Given a Hamilto- 
nian parameterized by X^"^' find param- 
eters u)^™'' which minimize the distance 
D[P\\Tuj("^AQ\('"^)]^ i-6- '^6 fiiid the point 
in the local orbit of Qa(,„) closest to P. 
The optimization of the parameter set a; = 
(ai, 6i, . . . , a AT, &Ar) is carried out in an itera- 
tive manner, i.e. we first optimize ai, then 6i, 
then 02, and so on, keeping previously opti- 
mized parameters fixed. This procedure is it- 
erated a number of times (typically in excess 
of 10 sweeps). Each parameter optimization is 
carried out using an iterative Monte Carlo pro- 
cedure, based on first randomly choosing test 
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values in the interval [0,1] and then choosing 
a sequence of nested intervals with decreasing 
the range from which more finely spaced subse- 
quent test values are drawn. 

3. Optimization of the Hamiltonian given a local 
transformation: Given a local transformation 
defined by parameters a;(™) find a set of param- 
eters a("+i) such that D[P\\T^_j^^-, (Qy^+i-,)] is 
minimized (subject to the constraint Q \ (m+i) G 
£k)- Similar to the procedure outlined above 
for the optimization of a; we optimize one of 
the parameters A at a time, carrying out a set 
of typically 10 — 20 or more sweeps over all 
parameters. Any one parameter is optimized 
by applying an deterministic search algorithm 
on the interval [—10,10] (with a suitable dis- 
cretization). The range of each parameter is 
therefore effectively truncated. The constraint 
Q\{m+i) € £k is taken into account by pre- 
setting the coefficients of ^-particle terms, £ > k 
to zero. For a three-particle system for exam- 
ple we have A = (ai,a2,a3,/3i2,/323,/3i3,7); see 



Eq. P^ . If the wish to compute differences say 
from £2 we would set 7 = from the start, and 
only optimize the remaining entries in A. 

4. This procedure is then iterated, i.e., goto 2. 

We would like to stress that no claim is made that 
this algorithm provides an exact result for Ck, this 
may not even be the case in the limit of an infinite 
number of iterations. The numerical scheme is an 
approximate procedure, providing at least an upper 
bound on Ck- It is also interesting to note that the al- 
gorithm contains a certain stochastic element (rooted 
in the Monte-Carlo optimization of the parameters lj 
as described above). We have found that it can be 
beneficial to allow occasional increases in the esti- 
mates of distance (i.e. 'uphill' motion), as this pre- 
vents dynamical arrest in local minima. Results re- 
ported are the minimal estimate of distance obtained 
during any one run of the algorithm, and not neces- 
sarily the distance estimate at the end of the nested 
set of iterations. 



[1] J. P. Crutchfield, Physica D 75, 11 (1994). 

[2] J. P. Crutchfield and K.Young, Phys. Rev. Lett. 63, 
105(1989). 

[3] Y. Bar- Yam, Complexity 9, 15 (2004). 

[4] D.J. Chalmers, Strong and Weak Emergence, The 
Re-Emergence of Emergence, Oxford University 
Press, Oxford UK (2002). 

[5] R. C. Ball, M. Diakonova, R. S. MacKay, Adv. 
Compl. Sys. 13, 327 (2010). 

[6] M. Diakonova and R. S. MacKay, Int. J. Bif. and 
Chaos 21 2297 (2011) 

[7] R.S. MacKay, Indecomposable coupled map lattices 
with non-unique phase, in: Dynamics of coupled map 
lattices and of related spatially extended systems, eds 
Chazottes J-R, Fernandez B, Lect Notes Phys 671, 
65-94 (Springer, 2005). 

[8] R.S. MacKay, Nonlinearity 21, T273 (2008). 

[9] K. Lindgren and M. G. Nordahl, Complex Systems 
2 409 (1988). 
[10] T. Kahle, E. Olbrich, J. lost, and N. Ay, Phys. Rev. 

E 79, 026201 (2009). 
[11] N. Ay, E. Olbrich, N. Bertschinger, and J. Jost. A 
Unifying Framework for Complexity Measures of Fi- 
nite Systems, Proceedings ECCS'06 (2006). Santa Fe 
Institute Working Paper 06-08-028. 
[12] S. Amari, IEEE Trans. Inf. Theory 47, 1701 (2001). 



[13] E. Olbrich, T. Kahle, N. Bertschinger, N. Ay, and J. 
Jost, arXiv:0912.4450 

[14] L. Steiner and T. Kahle Computing Informa- 
tion Projections Iteratively with, CIPI, available 
at https://github.com/tomlll/cipi. (accessed Au- 
gust 2010) 

[15] For reviews see M. Plenio and S. Virmani, Quant. 
Inf. Comp. 7, 1 (2007). R. Horodecki, P. Horodecki, 
M. Horodecki, and K. Horodecki, Rev. Mod. Phys. 
81, 865 (2009); O. Giihne and G. Toth, Phys. Re- 
ports 474, 1 (2009). 

[16] D. L. Zhou, Phys. Rev. A 80, 022113 (2009). 

[17] N. Linden, S. Popescu, and W. K. Wootters, Phys. 
Rev. Lett. 89, 207901 (2002). 

[18] D. L. Zhou, Phys. Rev. Lett. 101, 180505 (2008). 

[19] D. L. Zhou, B. Zeng, Z. Xu, and L. You, Phys. Rev. 
A 74, 052110 (2006). 

[20] T.M. Cover and J. A. Thomas Elements of Informa- 
tion Theory John Wiley & Sons (2006). 

[21] N. Ay and A. Knauf, Kybernetika 42, 517 (2007). 

[22] K. Kaneko, Chaos 2, 279 (1992). 

[23] S. Jalan, F. Atay, and J. Jost, Detecting global prop- 
erties of coupled dynamics using local symbolic dy- 
namics. Chaos 16, 1 (2006). 



